YAHSS

Yet Another High Scalability Story

dani(dot)caba at gmail(dot)com

Who are you?

dcProfile := map[string]string{
  "name": "Daniel Caballero",
  "title": "Devops Engineer",
  "mail": "dani(dot)caba at gmail(dot)com",
  "company": &SchibstedPT,
  "previously_at": []company{&NTTEurope, &Semantix, &Oracle},
  "linkedin": http.Get("https://www.linkedin.com/in/danicaba"),
  "extra": "Gestión DevOps de Arquitecturas IT@LaSalle",
}

So... I work

... I (some kinda) teach

... I (try to) program...

... I (would like to) rock...

... and I live

So... I really value my time

And I really don't like to waste it resolving production alerts

I'd say more: I really hate being reactive...

What is Schibsted?

And SPT?

And YAMS?

Brief service history

Why not offline transformations?

Why not reusing alternatives?

imaginarium sch media

Why not CDN/edge transformations?

Self service capabilities

Why not before?

We were busy. We had a lot of fun... at a high cost

We are actually devops

No ops/support team

Why transformations in golang?

Infrastructure management

Code reviews

  • Raffle
  • Quality gate
  • Other bots

Continuous delivery

  • travis
  • fpm
  • hardened images
  • spinnaker

Stress testing

Configuration management

Archaius | Viper |

Local fork given lack of defaults support

Logs

Monitoring and alerting

And escalations using pagerduty

Real time monitoring

Distributed tracing

S2S resiliency

Secrets management

Penetration testing

HTTP router

Services communication

  • fargo
  • eureka
  • load balancing
  • hystrix

bi-image + libvips

Caching

Zipkin

Custom metrics

Gifs..

And rate limits..

go-kit/negron logrus logging

Integration tests execution

Datastore access

Graceful shutdowns

AWS SDK usage

Multiregion

Smoke tests

More elasticity to reduce costs

Extra compression

Currently jpg-turbo

Bringing the service closer to the business

More engines

Also in use by attachments and CVs. PDF conv makes sense. Bench already done Video

Actual transformation pipelines

Include current workflow

More adoption?

Better capacity management

Incomming queue and reusing cache if no capacity Better degradation but efficient ASG triggers

ApiGW replacement?

Zuul could be replaced by Krakend

Simulating dependencies failures

Why not docker/k8s?

  • Portal
  • Migration exercise
  • Local tests

gRPC?

Why not Service Meshes?

Why not Google Cloud?

Imageflow

And Cassandra?

And PaaS?

And prometheus?

Before closing...

Are you going to opensource it?

Price calculator

  • Share price comparison

Other projects

Great thanks...

  • Sch*
  • Edge colleagues

Other Qs?